Repository navigation
fix(inference): replace unhealthy proxies and refresh linked models - #157
Conversation
|
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Audit completed on exact head bf68fd90c4ac656c55cffe9d255e3e1b13b18acc against 1857cd2554f068a0c75ac5f109a07afe9398ba16; no actionable source defect found. Reviewed unhealthy/current-version replacement, failed-start and upgrade rollback, signal recovery, catalog refresh/probe invalidation, lock ordering, and process/scope cleanup.
All 11 focused tests across InferenceProxy, routes, and HTTP passed. The new lifecycle regression was independently verified to fail against base production source (no stop for an unhealthy current-version child) and pass again after restoring this exact head. Both changed files passed formatting and lint.
The author's native macOS binary/process-group experiment and private/live-model requests were not independently rerun here. The existing Node spawner and desktop group-termination code were inspected. Fork CI is still running; merge remains gated on its final Fork Check, Agent Context Budget, and all other applicable checks.
A Mac Studio had CLIProxyAPI 8.0.77 installed, but an orphaned 8.0.76 process still answered on the inference port. The local Codex route rejected GPT-6.2-Astra with HTTP 503 because the older process did not advertise the model. The CLIProxyAPI updater normally stops its child before starting the replacement. If desktop shutdown escalated to terminating the backend process group, the detached proxy could survive it.
Start the proxy in its backend's process group so desktop group termination covers it as well. The existing child scope still handles ordinary proxy updates. An explicit Update verifies the running proxy even when its installed version is current. It replaces an unresponsive child, and reports failure if the replacement cannot start. Once the replacement starts, T3 refreshes its local model catalog and invalidates cached route probes for the previous process. The recovery watcher also handles signal exits.
Verified on macOS using the real 8.0.76 and 8.0.77 binaries on an isolated loopback port. With the previous settings, the proxy survived forced backend-group termination. With the change, both binaries exited and the replacement could serve on the same port. On the affected installation, after stopping the verified orphan and refreshing the catalog, two real model requests returned HTTP 200 and a T3 thread completed on the same provider and model. After recovery, the original affected thread also resumed and completed a shell command.
Implemented and verified with GPT-6 through Codex.